Identifying an Object in a Field of View

ABSTRACT

The technology disclosed relates to identifying an object in a field of view of a camera. In particular, it relates to identifying a display in the field of view of the camera. This is achieved by monitoring a space including acquiring a series of image frames of the space using the camera and detecting one or more light sources in the series of image frames. Further, one or more frequencies of periodic intensity or brightness variations, also referred to as ‘refresh rate’, of light emitted from the light sources is measured. Based on the one or more frequencies of periodic intensity variations of light emitted from the light sources, at least one display that includes the light sources is identified.

RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.16/295,663, entitled “IDENTIFYING AN OBJECT IN A FIELD OF VIEW”, filedMar. 7, 2019, (Attorney Docket No. ULTI 1040-5), which is a continuationof U.S. application Ser. No. 15/457,943 entitled “IDENTIFYING AN OBJECTIN A FIELD OF VIEW”, filed Mar. 13, 2017 (Attorney Docket No. LEAP1040-4/LPM-025US1), which is a continuation of U.S. application Ser. No.14/214,584, entitled “IDENTIFYING AN OBJECT IN A FIELD OF VIEW”, filedMar. 14, 2014 (Attorney Docket No. LEAP 1040-2/LPM-025US), which claimsthe benefit of U.S. provisional Patent Application No. 61/791,151,entitled, “DETERMINING THE POSITION OF A COMPUTER SCREEN,” filed 15 Mar.2013 (Attorney Docket No. LEAP 1040-1/LPM-025PR). The provisionalapplication is hereby incorporated by reference for all purposes.

FIELD OF THE TECHNOLOGY DISCLOSED

The technology disclosed relates generally to systems and methods fordetermining the positions objects in three-dimensional space, and morespecifically to determining the position of a computer screen ordisplay.

BACKGROUND

Existing systems allow a user to interact with a computer byinterpreting the position or movement of the user's body (or any otherobject) as a user-input command. The system responds to the commands ofthe user in accordance with software executing on the computer (e.g., anapplication or game) and displays the results of the received command ona computer screen (referred to herein also as a computer display). Forexample, the screen can display a representation of a button; the user“presses” the button by, for example, making a forward-moving gesturewith a finger. In some instances, however, the system can misinterpret agesture and perform an action unintended by the user. For example, thesystem can interpret a gesture as a button press even though the userdid not intend such an action. A need therefore exists for a systems andmethods that improve the accuracy of interpreting user gestures.

SUMMARY

The technology disclosed relates to identifying an object in a field ofview of a camera. In particular, it relates to identifying a display inthe field of view of the camera. This is achieved by monitoring a spaceincluding acquiring a series of image frames of the space using thecamera and detecting one or more light sources in the series of imageframes. Further, one or more frequencies of periodic intensity orbrightness variations, also referred to as ‘refresh rate’, of lightemitted from the light sources is measured. Based on the one or morefrequencies of periodic intensity variations of light emitted from thelight sources, at least one display that includes the light sources isidentified.

In various implementations, the technology disclosed also providessystems and methods for determining the position of a computer screenwith respect to a user or other objects in proximity to the screen.Light emitted by a light source can be captured by a camera; based onthe frequency of the light, it can be determined by a screen (bydetecting, e.g., the screen's refresh rate). The shape of the screen (orthe device in which it is mounted/housed) can further be used toidentify the screen. In one implementation, a user input can be used toidentify the screen. Once the screen is identified, knowledge of itsposition (and that of a user) can be used to improve the accuracy withwhich user gestures are interpreted.

In one aspect, a method of identifying a computer display in a field ofview of a camera includes acquiring a series of image frames from thecamera, detecting the presence of a light source in the series ofimages, analyzing the light source across the series of image frames forvariations in brightness, and identifying a frequency of the variationsas a refresh rate of the computer display.

The rate of capture of the series of image frames can be varied; therate can be 50, 60, 75, 100, or 120 frames per second. A distancebetween a user in the field of view of the camera and the computerdisplay can be detected, and a user-input command can be assigned to agesture of the user based at least in part on the distance. The presenceof a rectangular object in one of the image frames can be detected andthe rectangular object can be identified as the computer display.Identifying rectangular object as the computer display can includecomparing a length, width, or aspect ratio of the rectangular object toa known length, width, or aspect ratio of the computer display. A usercan be prompted to touch the computer display and a contact of theuser's hand with the computer display can be detected. The user can beprompted to touch the computer display a plurality of times and a planeoccupied by the computer display can be derived based on the detectedcontact. A user can be prompted to point, with a stylus, to a targetdisplayed on the computer display and a position of the stylus can bedetected. The user can be prompted to point multiple times at multipletargets and a plane occupied by the computer display can be derivedbased on the detected position of the stylus. The presence of a secondlight source can be detected and the light source can be identified asan incandescent bulb, halogen bulb, or fluorescent bulb.

In another aspect, a system of identifying a computer display in a fieldof view of a camera includes a camera for capturing a series of imagesof a field of view comprising the computer display and a location modulefor detecting the presence of a light source in the series of images,analyzing the light source across the series of image frames forvariations in brightness, and identifying a frequency of the variationsas a refresh rate of the computer display. An image-analysis module candetect a distance between a user in the field of view of the camera andthe computer display. The image-analysis module can assign a user-inputcommand to a gesture of the user based at least in part on the distance.

Reference throughout this specification to “one example,” “an example,”“one implementation,” or “an implementation” means that a particularfeature, structure, or characteristic described in connection with theexample is included in at least one example of the present technology.Thus, the occurrences of the phrases “in one example,” “in an example,”“one implementation,” or “an implementation” in various placesthroughout this specification are not necessarily all referring to thesame example. Furthermore, the particular features, structures,routines, steps, or characteristics can be combined in any suitablemanner in one or more examples of the technology. The headings providedherein are for convenience only and are not intended to limit orinterpret the scope or meaning of the claimed technology.

Advantageously, these and other aspects enable machines, computersand/or other types of intelligent devices, and/or other types ofautomata to obtain information about objects, events, actions, and/orusers employing gestures, signals, and/or other motions conveyingmeaning and/or combinations thereof. These and other advantages andfeatures of the implementations herein described, will become moreapparent through reference to the following description, theaccompanying drawings, and the claims. Furthermore, it is to beunderstood that the features of the various implementations describedherein are not mutually exclusive and can exist in various combinationsand permutations.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to like partsthroughout the different views. Also, the drawings are not necessarilyto scale, with an emphasis instead generally being placed uponillustrating the principles of the technology disclosed. In thefollowing description, various implementations of the technologydisclosed are described with reference to the following drawings, inwhich:

FIG. 1 illustrates an exemplary motion-capture system in accordance withimplementations of the technology disclosed.

FIG. 2 illustrates an exemplary computer system for image processing,analysis, and display in accordance with implementations of thetechnology disclosed.

FIG. 3 illustrates an exemplary display intensity and captured samplesthereof in accordance with implementations of the technology disclosed.

FIG. 4 illustrates a flowchart for identifying a computer display inaccordance with implementations of the technology disclosed.

FIG. 5 is a flowchart showing of a method of identifying an object in afield of view of camera.

DESCRIPTION

As used herein, a given signal, event or value is “responsive to” apredecessor signal, event or value of the predecessor signal, event orvalue influenced by the given signal, event or value. If there is anintervening processing element, action or time period, the given signal,event or value can still be “responsive to” the predecessor signal,event or value. If the intervening processing element or action combinesmore than one signal, event or value, the signal output of theprocessing element or action is considered “dependent on” each of thesignal, event or value inputs. If the given signal, event or value isthe same as the predecessor signal, event or value, this is merely adegenerate case in which the given signal, event or value is stillconsidered to be “dependent on” the predecessor signal, event or value.“Dependency” of a given signal, event or value upon another signal,event or value is defined similarly.

As used herein, the “identification” of an item of information does notnecessarily require the direct specification of that item ofinformation. Information can be “identified” in a field by simplyreferring to the actual information through one or more layers ofindirection, or by identifying one or more items of differentinformation which are together sufficient to determine the actual itemof information. In addition, the term “specify” is used herein to meanthe same as “identify.”

Motion-capture systems generally include (i) a camera for acquiringimages of an object; (ii) a computer for processing the images toidentify and characterize the object; and (iii) a computer display fordisplaying information related to the identified/characterized object. Alight source can also be included to illuminate the object. FIG. 1illustrates an exemplary motion-capture system 100. The system 100includes one or more light-capturing devices 102 (e.g., digital camerasor similar devices), each including an image sensor (e.g., a CCD or CMOSsensor) and an associated imaging optic (e.g., a lens). Any number ofcameras 102 can be arranged such that their fields of view overlap in aviewed region. One or more light-emitting devices 104 can be used toilluminate objects 106, 107 in the field of view. The cameras 102provide digital image data to a computer 108, which analyzes the imagedata to determine the 3D position, orientation, and/or motion of theobjects 106, 107 the field of view of the cameras 102. One of theobjects can be a computer display 106 in wireless or wired communicationwith the computer 108; as explained in greater detail below, thecomputer can generate images for the display 106 based at least in parton the position or motion of it or other objects in the field of view ofthe cameras 102 (e.g., the hand 107 of a user).

The cameras 102 can be, e.g., visible-light cameras, infrared (IR)cameras, ultraviolet cameras, or cameras operating in any otherelectromagnetic frequency regime. Preferably, the cameras 102 arecapable of capturing video images (i.e., successive image frames at aconstant rate of, say, fifteen frames per second, although no particularframe rate is required). The particular capabilities of cameras 102 canvary as to frame rate, image resolution (e.g., pixels per image), coloror intensity resolution (e.g., number of bits of intensity data perpixel), focal length of lenses, depth of field, etc. In general, for aparticular application, any cameras capable of focusing on objectswithin a spatial volume of interest can be used. For instance, tocapture motion of the hand of an otherwise stationary person, the volumeof interest might be a cube of one meter in length. To capture motion ofa running person, the volume of interest might have dimensions of tensof meters in order to observe several strides.

The cameras can be oriented in any convenient manner. In oneimplementation, the optical axes of the cameras 102 are parallel, butthis orientation is not required. As described below, each camera 102can be used to define a “vantage point” from which the object 106 isseen; if the location and view direction associated with each vantagepoint are known, the locus of points in space that project onto aparticular position in the camera's image plane can be determined. Insome implementations, motion capture is reliable only for objects in anarea where the fields of view of cameras 102 overlap; the cameras 102can be arranged to provide overlapping fields of view throughout thearea where motion of interest is expected to occur. In otherimplementations, the system 100 can include one or more light sources104, and the cameras 102 measure the reflection of the light emitted bythe light sources on objects 106. The system can include, for example,two cameras 102 and one light source 104; one camera 102 and two lightsources 104; or any other appropriate combination of light sources 104and cameras 102.

The computer 108 can generally be any device or combination of devicescapable of processing image data using techniques described herein. FIG.2 is a simplified block diagram of a suitably programmed general-purposecomputer 200 implementing the computer 108 according to an exemplaryimplementation of the technology disclosed. The computer 200 includes aprocessor 202 with one or more central processing units (CPUs), volatileand/or non-volatile main memory 204 (e.g., RAM, ROM, or flash memory),one or more mass storage devices 206 (e.g., hard disks, or removablemedia such as CDs, DVDs, USB flash drives, etc. and associated mediadrivers), a display device 208 (e.g., a liquid crystal display (LCD)monitor), user input devices such as keyboard 210 and mouse 212, and oneor more buses 214 (e.g., a single system bus shared between allcomponents, or separate memory and peripheral buses) that facilitatecommunication between these components.

The cameras 102 and/or light sources 104 can connect to the computer 200via a universal serial bus (USB), FireWire, or other cable, orwirelessly via Bluetooth, Wi-Fi, etc. The computer 200 can include adevice interface 216, implemented in hardware (e.g., as part of a USBport) and/or software (e.g., executed by processor 202), that enablescommunication with the cameras 102 and/or light sources 104. The camerainterface 216 can include one or more data ports and associated imagebuffers for receiving the image frames from the cameras 102; hardwareand/or software signal processors to modify the image data (e.g., toreduce noise or reformat data) prior to providing it as input to amotion-capture or other image-processing program; and/or control signalports for transmit signals to the cameras 102, e.g., to activate ordeactivate the cameras, to control camera settings (frame rate, imagequality, sensitivity, etc.), or the like.

The main memory 204 can be used to store instructions to be executed bythe processor 202, conceptually illustrated as a group of modules. Thesemodules generally include an operating system (e.g., Microsoft WINDOWSoperating system, the Unix operating system, the Linux operating system,the Xenix operating system, the IBM AIX operating system, the HewlettPackard UX operating system, the Novell NETWARE operating system, theSun Microsystems SOLARIS operating system, the OS/2 operating system,the BeOS operating system, the MAC OS operating system, the APACHEoperating system, an OPENACTION or OPENACTION operating system, iOS,Android or other mobile operating systems, or another operating systemplatform) that directs the execution of low-level, basic systemfunctions (such as memory allocation, file management, and the operationof mass storage devices), as well as higher-level software applicationssuch as, e.g., a motion-capture (mocap) program 218 for analyzing thecamera images to track the position of an object of interest and/or amotion-response program for computing a series of output images (oranother kind of response) based on the tracked motion. Suitablealgorithms for motion-capture program are described further below aswell as, in more detail, in U.S. patent application Ser. No. 13/414,485,filed on Mar. 7, 2012 and Ser. No. 13/742,953, filed on Jan. 16, 2013,and U.S. Provisional Patent Application No. 61/724,091, filed on Nov. 8,2012, which are hereby incorporated herein by reference in theirentirety. The various modules can be programmed in any suitableprogramming language, including, without limitation high-level languagessuch as C, C++, C#, OpenGL, Ada, Basic, Cobra, FORTRAN, Java, Lisp,Perl, Python, Ruby, or Object Pascal, or low-level assembly languages.

The memory 204 can further store input and/or output data associatedwith execution of the instructions (including, e.g., input and outputimage data 220) as well as additional information used by the varioussoftware applications; for example, in some implementations, the memory204 stores an object library 222 of canonical models of various objectsof interest. As described below, an object detected in the camera imagescan identified by matching its shape to a model in the object library222, and the model can then inform further image analysis, motionprediction, etc.

In various implementations, the motion captured in a series of cameraimages is used to compute a corresponding series of output images fordisplay on the (computer) display screen 208. For example, camera imagesof a moving hand can be translated into a wire-frame or other graphicdepiction of the hand by the processor 202. Alternatively, hand gesturescan be interpreted as input used to control a separate visual output; byway of illustration, a user can be able to use upward or downwardswiping gestures to “scroll” a webpage or other document currentlydisplayed, or open and close her hand to zoom in and out of the page. Inany case, the output images are generally stored in the form of pixeldata in a frame buffer, which can, but need not be, implemented in mainmemory 204. A video display controller reads out the frame buffer togenerate a data stream and associated control signals to output theimages to the display 208. The video display controller can be providedalong with the processor 202 and memory 204 on-board the motherboard ofthe computer 200, and can be integrated with the processor 202 orimplemented as a co-processor that manipulates a separate video memory.In some implementations, the computer 200 is equipped with a separategraphics or video card that aids with generating the feed of outputimages for the display 208. The video card generally includes agraphical processing unit (“GPU”) and video memory, and is useful, inparticular, for complex and computationally expensive image processingand rendering. The graphics card can implement the frame buffer and thefunctionality of the video display controller (and the on-board videodisplay controller can be disabled). In general, the image-processingand motion-capture functionality of the system can be distributedbetween the GPU and the main processor 202 in various conventional waysthat are well characterized in the art.

The computer 200 is an illustrative example; variations andmodifications are possible. Computers can be implemented in a variety ofform factors, including server systems, desktop systems, laptop systems,tablets, smart phones or personal digital assistants, and so on. Aparticular implementation can include other functionality not describedherein, e.g., wired and/or wireless network interfaces, media playingand/or recording capability, etc. In some implementations, one or morecameras can be built into the computer rather than being supplied asseparate components. Further, the computer processor can be ageneral-purpose microprocessor, but depending on implementation canalternatively be, e.g., a microcontroller, peripheral integrated circuitelement, a customer-specific integrated circuit (“CSIC”), anapplication-specific integrated circuit (“ASIC”), a logic circuit, adigital signal processor (“DSP”), a programmable logic device such as afield-programmable gate array (“FPGA”), a programmable logic device(“PLD”), a programmable logic array (“PLA”), smart chip, or other deviceor arrangement of devices.

Further, while the computer 200 is described herein with reference toparticular blocks, this is not intended to limit the technologydisclosed to a particular physical arrangement of distinct componentparts. For example, in some implementations, the cameras 102 areconnected to or integrated with a special-purpose processing unit that,in turn, communicates with a general-purpose computer, e.g., via directmemory access (“DMA”). The processing unit can include one or more imagebuffers for storing the image data read out from the camera sensors, aGPU or other processor and associated memory implementing at least partof the motion-capture algorithm, and a DMA controller. The processingunit can provide processed images or other data derived from the cameraimages to the computer for further processing. In some implementations,the processing unit sends display control signals generated based on thecaptured motion (e.g., of a user's hand) to the computer, and thecomputer uses these control signals to adjust the on-screen display ofdocuments and images that are otherwise unrelated to the camera images(e.g., text documents or maps) by, for example, shifting or rotating theimages.

In one implementation, a location module 224 identifies an object as thedisplay 208. the display 208 can be positioned within the field of viewof the cameras 102; in accordance with the flowchart 400 illustrated inFIG. 4, images from the camera 102 are received by the device interface216 (action 402) can thus include representations of the display 208.The display 208 can display images on its screen by periodicallyrefreshing the screen (i.e., displaying a first frame on the screen,waiting a certain amount of time, displaying a second frame, and so on).This periodic refreshing is known as the refresh rate of the display208. In some implementations, the refresh rate is the same as the framerate of the display; in other implementations, the two rates differ(i.e., one frame is refreshed more than once). As the term is usedherein, the refresh rate is the rate at which a display's intensity ofemitted light rises and falls, whatever mechanism or system causes it todo so. Displays created using any number of technologies (e.g.,cathode-ray tubes, plasma, LCD, or LED displays) can utilize this methodof updating. Because of the refresh rate, the intensity or luminosity ofthe light emitted by the display 208 and captured by the camera 102 canvary; if, for example, an image is captured during or soon after arefresh, the intensity of the light captured is high. If, on the otherhand, an image is captured just before a next refresh of the display,the intensity can be low. The presence of a light source can be detectedin the series of images (action 404) by, for example, comparing theintensities of areas of each image and/or by detecting display-likeshapes in the series of images.

FIG. 3 illustrates an exemplary chart 300 showing the intensity 302 oflight emitted by the display 208. As the display 208 refreshes (as,e.g., its LED drivers fire) the intensity 302 increases; as time passes,however, the intensity decreases until a next refresh. The intensity 302is depicted as a saw-tooth wave in FIG. 3, but the intensity 302 canhave any shape (e.g., a sine wave, square wave, pulse with exponentialdecay, etc.).

The intensity 302 of the display 208 is captured by the camera 102 as aseries of samples 304, in accordance with a frame rate of the camera102, taken from a series of images captured therefrom. The series ofimages can be a series of temporally sequential images that include thedisplay 208 and are taken at a certain number of frames per second, suchas 50, 60, 75, 100, or 120 frames per second. If a sample 304 is takenat a high point on the intensity curve 302, it can receive a high value,and if the sample is taken at a low point on the curve 302, it canreceive a low value. Any method of determining the samples 304 is withinthe scope of the technology disclosed such as, for example, analyzingthe pixels in an acquired series of images corresponding to the display208 and determining their intensity values.

Analysis of the samples 304 (using, for example, the image-analysismodule 218, the location module 224, or similar variants thereof) yieldsa frequency of the intensity curve 302 (action 406). In oneimplementation, the peaks (or any other distinguishing feature) of theintensity curve 302, from which the frequency of the curve 302 can becomputed. In another implementation, the beat frequency between thefrequency of the intensity curve 302 and the frequency of the samplerate of the samples 304 is determined by analyzing the samples 304;given the beat frequency and the frequency of the sample rate (i.e., theframes-per-second of the camera 102), the frequency of the intensitycurve can be computed.

The frequency of the intensity curve 302 can be used to identify theobject producing the sample curve 302 as a computer display (action408). For example, if the frequency is 60 Hz or 72 Hz, the object isdeemed to be a computer display because those are common display refreshrates. The system 200 can maintain a list of recognized refresh rates inthe memory 204 and compare the frequencies thereto. In oneimplementation, the system can tolerate a deviation of 1%, 2%, or 5% inthe computed frequency to a recognized refresh rate and still deem amatch. The list of refresh rates can be updated by a user or by asoftware update or patch.

In one implementation, the image-analysis module 218 determines a shapefor an object detected in a field of view of the camera 102. A lengthand width of the object can be determined in accordance with the methodsand systems described above; if the length and width fall within a rangeof accepted lengths and widths for computer displays, the system 200 candeem the object to be a computer display. The system 200 can analyzeother attributes of the object (such as its aspect ratio and/orplacement in the field of view) in making this determination. Thephysical attributes of the object can be used to differentiate theobject from another type of periodically fluctuating light source, suchas an incandescent, fluorescent, or halogen bulb.

The location module 224 can prompt the user to assist in determining andlocating the display 208. In one implementation, the location moduledisplays a graphical or textual prompt on the display 208 instructingthe user to touch the display; the image-analysis module 218 can trackthe motion of the user (e.g., the hand of the user) using the systemsand methods described above and detect when the user's hand has madecontact with the display 208. At that time, the system 200 deems theobject being touched to be the display 208. In another implementation,the location module 224 displays a target on the display 208 and promptsthe user to point a stylus or similar object at the target; by detectingthe angle and position of the stylus, the location module 224 deems theobject being pointed at to be the display 208. In relatedimplementations, the user can touch or point to multiple (e.g., three)locations on the display 208, from which the location module 224 candeduce the plane in which the display 208 lies.

Once the display 208 is identified, its position can be determinedrelative to the system 200 and/or to a user. The distance between thedisplay 208 and the user can be used to accept, decline, or modifygestures of the user as user-input commands. For example, if the usergestures toward the display 208 in a button-pressing motion, the motioncan be interpreted as a first user-input command at one distance and asecond user-input command at a second distance. The interpretation ofthe user gesture can further depend on the amount of distance over whichthe gesture travels; a small motion can register as a user-input commandat a small user-to-screen distance while the same small motion may notregister as a user-input command at a large distance. For example, inorder to make a button-pressing gesture, a user can need to make his orher finger travel over a greater distance when the user is farther fromthe display 208; at a smaller user-to-display distance, a lesserdistance of travel triggers button-pressing user-input command.

FIG. 5 is a flowchart showing of a method of identifying an object in afield of view of camera. Flowchart 500 can be implemented at leastpartially with and/or by one or more processors configured to receive orretrieve information, process the information, store results, andtransmit the results. Other implementations may perform the actions indifferent orders and/or with different, fewer or additional actions thanthose illustrated in FIG. 5. Multiple actions can be combined in someimplementations. For convenience, this flowchart is described withreference to the system that carries out a method. The system is notnecessarily part of the method.

At action 502, a space is monitored using a camera. This is accomplishedby capturing a series of image samples of the space in accordance with aframe rate of the camera, according to one implementation. The series ofimages can be a series of temporally sequential images that include alight source such as a display and are taken at a certain number offrames per second, such as 50, 60, 75, 100, or 120 frames per second, asdiscussed above.

At action 504, one or more frequencies of periodic intensity orbrightness variations of light emitted from the light sources (refreshrate) is measured in response to detection of one or more light sourcesin the space. In one implementation, a beat frequency of the periodicbrightness variations is used against the capture rate to determine therefresh rate, as discussed above.

At action 506, at least one object is identified within the space thatincludes the light sources based on the measured frequencies of periodicintensity variations. In some implementations, a shape of the identifiedobject is determined along with a length, width, or aspect ratio of theidentified object, as discussed above. In one implementation, the objectis identified as a display when the shape of the identified object isrectangular. Some implementations of identifying the object as a displayfurther include comparing a length, width, or aspect ratio of theidentified object to a predetermined length, width, or aspect ratio ofthe display. In other implementations, the object is identified as anincandescent bulb, halogen bulb, or fluorescent bulb based on thedetermined shape.

In some implementations, a distance between a user in the field of viewof the camera and the object is determined, as discussed above. Further,a context is set for interpreting user gestures as user-input commandsbased at least in part on the determined distance. In otherimplementations, a user is prompted to make a plurality of contacts atmultiple locations on the display, based on which a plane occupied bythe display is determined. In yet other implementations, the user isprompted to point, with a stylus, to a target presented on the display,based on which a position of the stylus is calculated. Further, the useris prompted to point multiple times at multiple targets, responsive towhich a plane occupied by the display is determined based on thedetected positions of the stylus.

It should also be noted that implementations of the technology disclosedcan be provided as one or more computer-readable programs embodied on orin one or more articles of manufacture. The article of manufacture canbe any suitable hardware apparatus, such as, for example, a floppy disk,a hard disk, a CD ROM, a CD-RW, a CD-R, a DVD ROM, a DVD-RW, a DVD-R, aflash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general,the computer-readable programs can be implemented in any programminglanguage. Some examples of languages that can be used include C, C++, orJAVA. The software programs can be further translated into machinelanguage or virtual machine instructions and stored in a program file inthat form. The program file can then be stored on or in one or more ofthe articles of manufacture.

Certain implementations of the technology disclosed were describedabove. It is, however, expressly noted that the technology disclosed isnot limited to those implementations, but rather the intention is thatadditions and modifications to what was expressly described herein arealso included within the scope of the technology disclosed. Moreover, itis to be understood that the features of the various implementationsdescribed herein were not mutually exclusive and can exist in variouscombinations and permutations, even if such combinations or permutationswere not made express herein, without departing from the spirit andscope of the technology disclosed. In fact, variations, modifications,and other implementations of what was described herein will occur tothose of ordinary skill in the art without departing from the spirit andthe scope of the technology disclosed. As such, the technology disclosedis not to be defined only by the preceding illustrative description.

What is claimed is:
 1. A method of recognizing a display in a field ofview of a camera, the method including: acquiring a series of imageframes from the camera, the image frames including capturing an objectwithin a field of view of the camera; and obtaining an indication thatthe object in the series of images is recognized as a display within thefield of view of the camera that includes one or more light sources, therecognizing based on a determined rate of periodic change in brightnessof the one or more light sources detected in the series of image frames.2. The method of claim 1, wherein the rate of periodic change inbrightness is determined by measuring frequencies of the one or morelight sources over time.
 3. The method of claim 1, further includingvarying a rate of capture of the series of image frames.
 4. The methodof claim 3, wherein the rate of capture is 50, 60, 75, 100, or 120frames per second.
 5. The method of claim 3, further including using abeat frequency of periodic brightness variations against the rate ofcapture to determine the rate of periodic change in brightness; whereinthe one or more light sources are analyzed across the series of imagesfor periodic brightness variations.
 6. The method of claim 1, furtherincluding determining a distance between a user in the field of view ofthe camera and the display.
 7. The method of claim 6, further includingsetting a context for interpreting user gestures as user-input commandsbased at least in part on the determined distance.
 8. The method ofclaim 1, further including prompting a user to make a plurality ofcontacts at multiple locations on the display and determining a planeoccupied by the display based on the plurality of contacts.
 9. Themethod of claim 1, further including prompting a user to point, with astylus, to a target presented on the display and detecting a position ofthe stylus.
 10. The method of claim 9, further including prompting theuser to point multiple times at multiple targets and determining a planeoccupied by the display based on the detected position of the stylus.11. A method of recognizing a display in a field of view of a camera,the method including: monitoring using a camera, a space within thefield of view of the camera; and in response to a detection of one ormore light sources in the space, obtaining an indication that at leastone object within the space is recognized as a display that includes theone or more light sources, the recognizing based on a determined rate ofperiodic change in brightness of the one or more light sources detectedin the space.
 12. The method of claim 11, wherein the recognizing of theat least one object as the display includes determining a shape of theat least one object.
 13. The method of claim 12, further includingdetermining a length, a width, or an aspect ratio of the at least oneobject.
 14. The method of claim 11, wherein the recognizing of the atleast one object as the display includes determining that a shape of theat least one object is rectangular.
 15. The method of claim 11, whereinthe recognizing of the at least one object as the display includescomparing a length, a width, or an aspect ratio of the at least oneobject to a predetermined length, a predetermined width, or apredetermined aspect ratio of the display.
 16. A system of recognizing adisplay in a field of view of a camera, the system including: the camerathat captures a series of images including at least one object in thefield of view of the camera; and at least one processor coupled to thecamera and a computer readable storage medium storing instructions thatwhen executed implement: obtaining an indication that the at least oneobject is recognized as a display in the field of view of the camerathat includes one or more light sources, the recognizing based on adetermined rate of periodic change in brightness of the one or morelight sources detected in the series of images.
 17. The system of claim16, further configured to determine a distance between a user in thefield of view of the camera and the display.
 18. The system of claim 16,further configured to vary a rate of capture of the series of images.19. The system of claim 18, further configured to use a beat frequencyof the periodic change in brightness against the rate of capture todetermine the rate of periodic change in brightness.
 20. Anon-transitory computer readable storage medium storing computer programinstructions that, when executed by a processor perform the method ofclaim 1.